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3D models and (b) generating 2D drawings. We investigate whether framing prompts that target 
either of these practices enhance the effectiveness of an educational technology that supports 
collaborative translation among multiple representations. To this end, we conducted a quasi- 
experiment with 565 undergraduate chemistry students. All students collaboratively built phy- 
sical 3D models of molecules and translated them into 2D drawings. In a business-as-usual control 
condition, students drew on paper, without support from an educational technology. In two 
experimental conditions, students drew in an educational technology that provided feedback and 
prompted collaboration. One condition received framing prompts to focus on physical models 
(model condition); another received prompts to generate intermediary drawings on paper (draw 
condition). Compared to the control condition, the model condition showed higher learning 
gains, but the draw condition showed lower learning gains. Analyses of log data showed that 
students made more model-based errors, and the prompts in the model condition reduced these 
model-based errors. However, interviews with instructors showed that they prefer drawing-fo- 
cused prompts, in contrast to our results. These findings offer theoretical insights into how stu- 
dents learn to translate among representations. Furthermore, they yield practical reeommenda- 
tions for the use of educational technologies that support learning with multiple representations. 


1. Introduction 


Instructors play a crucial role in implementing educational technologies in classrooms. Although educational technologies can 
support students' learning by providing help in similar ways as human instructors would (VanLehn, 2011), instructors often provide 
additional support that frames students’ interactions with educational technologies. For instance, at the beginning of a class, in- 
structors may explain how the technology addresses instructional goals and how students should use it to engage with content. 
Throughout the class, instructors typically continue to encourage students to interact with the technology in a certain way. To date, 
we know little about how framing interactions with educational technologies affects students’ learning and hence lack practical 
recommendations for how to integrate educational technologies in instruction. 
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Typically, instructors frame interactions with technologies in a way that reflects how they help students interact with content. For 
instance, instruction in science, technology, engineering, and mathematics (STEM) domains often involves translating between 
physical 3D and virtual 2D representations (Ainsworth, 2008; Kozma, 2003). These translation activities are difficult for students 
because they must understand features in each representation and make connections between them (Ainsworth, 2008; Rau, 2017; 
Stull, Hegarty, Dixon, & Stieff, 2012). Hence, instructors typically frame students' interactions by focusing their attention on the 
aspects that are particularly difficult for them in translating among representations. 

Our focus on framing practices originates from a prior experiment (Rau, Bowman, & Moore, 2017). This experiment tested the 
effects of an educational technology in which students collaboratively translated physical 3D models of chemical molecules into 2D 
drawings. The educational technology served to provide feedback on students' 2D drawings and to help them work with peers to fix 
mistakes in their drawings. During the experiment, we observed two prevalent framing practices. First, we found that instructors 
often framed students’ interactions with the technology by focusing them on physical 3D models. Specifically, instructors prompted 
students to construct a 3D model and then translate it into a virtual 2D drawing within the educational technology, where they would 
receive feedback on the accuracy of their drawing. Second, instructors often framed students' interactions by focusing students on 2D 
drawings. Specifically, instructors prompted students to use the 3D model to generate an intermediary 2D drawing on paper before 
generating the virtual drawing to receive feedback from the educational technology. Which framing practice is most effective at 
enhancing students' learning of content knowledge from the collaborative translation activities? How do these framing practices 
affect students’ interactions with the technology? 

We address these questions in a quasi-experiment as part of an undergraduate chemistry course. We focused on a session of this 
course in which students collaboratively translated physical 3D ball-and-stick models into virtual 2D wedge-dash drawings to learn 
molecular geometry. At the beginning of the course session, we provided framing prompts that focused students' interactions on 
models or on intermediary drawings. We examined how framing prompts affected students’ learning of content knowledge, their 
problem-solving performance within the technology, and instructors' impressions of how students interacted with the representations 
and the technology. 


2. Theoretical background 


In the following, we first review prior research on students' difficulties in translating among multiple representations and in 
collaborating with peers. We highlight how educational technologies have been used to address each of these difficulties. Then, we 
discuss our prior work in which we investigated a technology to support translation and collaboration. We describe how instructors' 
framing practices may have affected students’ interactions with the technology. Finally, we review prior research on potential effects 
of the framing practices on students' learning, which yields the research questions that we investigate in a quasi-experiment. 


2.1. Students' difficulties in translating among multiple visual representations 


Instruction in STEM domains often asks students to translate among multiple visual representations to learn abstract concepts 
(Ainsworth, 2008; Kozma, 2003). For instance, to learn about the spatial arrangement of atoms underlying molecular geometry, 
students often construct physical 3D ball-and-stick models (Fig. 1A) and draw 2D drawings that use wedges and dashes to show the 
spatial arrangement of atoms within the molecule (Fig. 1B). Unless students can translate among these representations, they do not 
benefit from them (Ainsworth, 2008; Rau, Aleven, & Rummel, 2015; Rau, 2017). 

Yet, it is well documented that students have difficulties with such translations, which can severely impede their learning in STEM 
domains (Ainsworth, 2008; Rau, 2017; Stull et al., 2012). Translating requires students to map visual features of one representation to 
corresponding features in the other representation. To do so, students have to hold visual features in working memory and mentally 
rotate these features to align them (Hegarty & Waller, 2005). Because students with low spatial skills require more cognitive re- 
sources to perform mental rotation tasks, translation activities can induce cognitive overload that impedes their learning (Hegarty & 
Waller, 2005; Stieff, 2007). Consequently, difficulties in translating among representations are particularly pronounced for students 
with low spatial skills (H6ffler, 2010; Uttal et al., 2013). 

Prior research shows that educational technologies can help students overcome their difficulties in translating among multiple 
representations (Rau et al., 2015, 2017; Kozma & Russell, 2005; Seufert, 2003). Technologies can provide immediate feedback on 
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Fig. 1. Physical ball-and-stick model (A) and wedge-dash structure (B). Each shows two molecules with the same atoms but different 3D spatial 
arrangement of the atoms. 
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translation activities and prompt students to map specific visual features of representations (Rau et al., 2017; Seufert, 2003). For 
instance, when students make errors related to spatial orientation, the technology can prompt students to rotate a physical 3D model 
and map its features to a 2D virtual representation. Akin to the support provided by instructors, technologies can adapt to students' 
needs by directing attention to aspects of representations that students least understand (VanLehn, 2011). Such support in tech- 
nologies has been shown to help students translate between multiple representations and increase learning of content knowledge 
(Kozma & Russell, 2005; Seufert, 2003). 


2.2. Students' difficulties in collaborating 


In many STEM domains, students work collaboratively on problems that require translation among multiple representations 
because they can help each other map visual features across representations (van Dijk, Gijlers, & Weinberger, 2014; White & Pea, 
2011). Collaboration may engage students more actively in the translation activities (Chi, 2009). Students may realize that they hold 
divergent views on the representations, which may prompt them to engage more deeply in making sense of the translations (White & 
Pea, 2011; Zhang & Linn, 2011). 

However, students' difficulties in collaborating effectively are well documented (Lou, Abrami, & D'Apollonia, 2001; White & Pea, 
2011). Collaboration requires students to coordinate activities with peers, communicate information, and to co-construct meaning. 
Because this is cognitively demanding (Kirschner, Paas, & Kirschner, 2010), students may choose to work individually or divide the 
work, which reduces opportunities to learn from each other. 

To help students overcome these difficulties, instructional support can script students’ collaborative interactions. Such colla- 
boration scripts can guide students' interactions, for instance, by asking questions for students to discuss or by prompting them to 
explain concepts to one another (Fischer, Kollar, Stegmann, & Wecker, 2013; Vogel, Wecker, Kollar, & Fischer, 2016). Educational 
technologies have been shown to be particularly effective because they can provide adaptive collaboration scripts that tailor colla- 
boration support to students' needs. For instance, when students make an error, such scripts can prompt them to discuss the specific 
error with their partner. 

While collaboration scripts have been shown to enhance the quality of students’ collaboration (Vogel et al., 2016), research has 
shown mixed results for students' learning of content knowledge (for an overview, see Magnisalis, Demetriadis, & Karakostas, 2011; 
Vogel et al., 2016). Some studies found that adaptive collaboration scripts enhance learning of content knowledge (e.g., Karakostas & 
Demetriadis, 2011), but several studies have failed to show benefits for these scripts compared to non-adaptive collaboration scripts 
(Walker, Rummel, & Koedinger, 2009) or individual learning (Baghaei, Mitrovic, & Irwin, 2007). 


2.3. Technologies can support collaboration and translation among representations 


Our own prior research shows that adaptive collaboration scripts can enhance students’ learning of content knowledge from 
activities that require translation among multiple representations. Specifically, we investigated the effectiveness of an adaptive 
collaboration script on students' learning from activities that asked them to translate physical 3D ball-and-stick models into virtual 2D 
wedge-dash drawings (Rau et al., 2017). The script provided error-specific prompts to discuss translations when students made a 
mistake in translating among the representations. Results showed higher learning gains for students who received the adaptive 
collaboration script than for students who worked on the same activities without the script. 

Given that prior research has shown mixed results for the effectiveness of collaboration scripts on learning of content knowledge, 
it is important to investigate which aspects of our collaboration script account for its effectiveness. Our observations of the ex- 
periment suggest that the adaptive collaboration scripts may have been effective because they engaged students in collaborative 
interactions with the physical 3D models or the 2D virtual drawings. First, the script seemed to promote the use of physical models 
because it prompted students to discuss how the physical model depicts the spatial arrangement of atoms within the molecule. For 
example, if students incorrectly depicted the arrangement of atoms in their virtual wedge-dash drawing, the script prompted stu- 
dents: “Look at your physical model. Does your drawing on the screen resemble what you have built?” We observed that students 
often rotated their physical models to orient them in line with the virtual drawings. As detailed below, this is one of the two framing 
practices that we observed instructors to encourage. 

Second, the script seemed to promote the generation of additional drawings on paper when it prompted students to discuss how to 
fix the virtual wedge-dash drawing. For example, if students made an error in the atoms they drew virtually, the script prompted 
them: “No, this is not correct. Look at the chemical formula and discuss with your partner which atom symbols you have to draw.” We 
observed that students often generated paper-based drawings to fix their virtual drawings, which is, as detailed below, also a practice 
encouraged by instructors. 

It is possible that focusing students on either of these practices accounted for the effectiveness of the adaptive collaboration script 
for their learning of content knowledge. However, our prior experiment did not contrast whether focusing students’ interactions on 
the physical models or on the intermediary drawings is most effective. 


2.4. Framing practices reflect disciplinary practices with multiple representations 
As mentioned, we observed that instructors in our prior experiment framed students' interactions with the educational technology 
by focusing their attention on physical models or on generating paper-based drawings. These two framing practices correspond to 


disciplinary practices common in STEM domains (National Research Council, 2012). Prior research offers mixed views on whether 
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these two framing practices enhance or hinder learning, as discussed in detail below. 


2.4.1. Focusing on physical models 

First, prior research suggests that focusing students' attention on physical models can help them learn content knowledge (Pouw, 
van Gog, & Paas, 2014; Stull et al., 2012). For instance, interactions with ball-and-stick models (Fig. 1A) help students learn that the 
balls (atoms) have a designated number of holes for sticks (bonds) which are spread out as far apart as possible, resulting in a 
tetrahedral shape. Prior work shows that the physical action of rotating and orienting models helps students engage in translation 
between 3D and 2D models (Pouw et al., 2014; Stull et al., 2012). Further, physically rotating models may alleviate difficulties in 
mental rotation for students with low spatial skills (Hoffler, 2010; Pouw et al., 2014). Particularly, it may help students learn how to 
rotate the 3D ball-and-stick model for projection onto a 2D plane (Fig. 1), which has been shown to be particularly difficult for 
students in chemistry (Barrett, Stull, Hsu, & Hegarty, 2014; Stull et al., 2012). Thus, it is possible that focusing students' attention on 
the physical ball-and-stick models is particularly effective for students with low spatial skills or low prior knowledge about the 
content or the representations (Barrett & Hegarty, 2016; Héffler, 2010). 

However, some prior research suggests that focusing students' interactions on physical models could hinder learning because 
students often do not know how to spatially align physical models with other representations to map features (Barrett et al., 2014). 
Because students can freely rotate physical models, they may fail to spatially orient the models to facilitate their translation into 2D 
drawings. Given that this aspect of the translation task is cognitively demanding, focusing students' attention on the physical models 
may further increase cognitive load and thereby impede students' learning. For example, several studies show that students benefit 
less from rotating a physical 3D model themselves, compared to watching an instructor or a technology rotate the model (Barrett & 
Hegarty, 2016; Springer, 2014). Rotating and spatially aligning objects is particularly challenging for students with low spatial skills 
(Barrett & Hegarty, 2016). Therefore, focusing interactions on physical models may disadvantage students with low spatial skills. 


2.4.2. Focusing on drawings 

Second, instructors often focus students' attention on drawings because drawing is a prevalent professional practice (Evagorou, 
Erduran, & Mantyla, 2015; Frankel, 2005). For example, professional chemists often draw intermediary representations to help them 
translate among representations while they collaborate with colleagues (Kozma, 2003). Indeed, prior research shows that drawing 
representations can help students learn content knowledge (Brooks, 2009; Schmeck, Mayer, Opfermann, Pfeiffer, & Leutner, 2014; 
Zhang & Linn, 2011), particularly from translation activities in chemistry (Cooper, Stieff, & DeSutter, 2017; Zhang & Linn, 2011). 
Further, when focusing students' interactions on drawings, students often generate additional visual representations on paper. Prior 
research suggests that drawing on paper may be more effective than drawing on the computer, potentially because students are more 
familiar with drawing on paper (Leutner & Schmeck, 2014). Drawing on paper can also facilitate collaboration because students can 
easily share their drawings and quickly make changes to their drawings (Brooks, 2009; White & Pea, 2011). Further, such drawings 
may alleviate students' difficulties with mental rotation because students can physically rotate the paper to align with the physical 
model or with a different 2D drawing. Finally, because spatial skills do not seem to moderate students' benefit from drawing activities 
(Schmeck et al., 2014), focusing students’ interactions on drawings may equally benefit students with low and high spatial skills. 

However, prior research also shows that drawing activities are cognitively demanding, especially if the drawing task involves 
mental rotation (Schwamborn, Thillmann, Opfermann, & Leutner, 2011). As a result, students may focus on generating the drawing 
without processing and mapping features across representations. Because translation tasks are already more demanding for students 
with low spatial skills (H6éffler, 2010), focusing students' interactions on drawing may be ineffective for students with low spatial 
skills. 


3. Research questions 


Our prior study on an adaptive collaboration script for translating among representations (Rau et al., 2017) did not differentiate 
between the two framing practices of focusing students’ interactions on physical models or on intermediary drawings on paper. 
Therefore, we do not know which framing practice accounts for the effectiveness of the adaptive collaboration script and how 
instructors should frame students' interactions with the technology. 

To address this gap, we investigate the effects of framing practices in an adaptive collaboration script that helps students translate 
among multiple representations. Specifically, we added framing prompts that either focused students’ interactions on physical models 
or on intermediary drawings. Because the prompts served to frame the students' interactions, they were provided before students 
worked on the translation activities. 

Our first research goal is to investigate the effects of an adaptive collaboration script with framing prompts that focused on 
physical models or on intermediate drawings, when compared to a business-as-usual control condition without a collaboration script, 
which served as a baseline comparison. This control condition allowed us to test whether the framing prompts enhance or hinder 
learning. Specifically, we investigate: 

Research question 1: Are framing prompts more effective than business-as-usual in terms of enhancing students’ learning of 
content knowledge if they focus students! interactions on physical models or on intermediary drawings? Based on the prior research 
we reviewed above, one can argue for potentially positive and negative effects of each framing practice. However, because both 
prompts align with disciplinary practices that are valued in chemistry (National Research Council, 2012), we expect that both 
prompts are effective in enhancing learning of content knowledge, compared to business-as-usual. 

Further, prior research shows that translating between representations is particularly difficult for students with low spatial skills 
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(Hegarty & Waller, 2005; Stieff, 2007). Hence, we investigate whether students' spatial skills moderate the effects of model-focused 
and drawing-focused prompts, especially for students with low spatial skills (research question 1a). Based on prior research, one can 
argue for positive or negative effects of each framing prompt to be particularly pronounced for students with low spatial skills. 

Our second research goal is to investigate how the framing prompts affect students’ interactions with the technology. Because 
each framing prompt focused on a specific representation, we expect students to enhance problem-solving performance with that 
representation. To this end, we compared the types of errors that students made when using the adaptive collaboration script and 
assessed whether each framing prompt reduced errors related to modeling or drawing. Specifically, we investigate: 

Research question 2: Do model-focused and drawing-focused prompts enhance students' performance while working with the 
respective representation? We expect model-focused prompts to reduce errors related to modeling. Conversely, we expect drawing- 
focused prompts to reduce errors related to drawing. Furthermore, we investigate whether students' spatial skills moderate the effects 
of model-focused and drawing-focused prompts (research question 2a). 

Finally, our third research goal is to gain insights into whether instructors found these framing prompts to be effective. Hence, we 
conducted interviews with instructors to explore: 

Research question 3: What are instructors' impressions on the effects of the model-focused and drawing-focused prompts? 


4. Method 
4.1. Chemistry course and participants 


We addressed these questions in a quasi-experiment with 693 students enrolled in an introductory undergraduate chemistry 
course at a Midwestern U.S. university. The course was designed for first-year students with high prior chemistry and math 
knowledge and who intend to major in chemistry or similar STEM fields. 

The course involved two 50-min lectures attended by all students, one 50-min discussion session, and one 3-hr lab session each 
week. Lab and discussion sessions were held in smaller sections with approximately 20 students. The lab and discussion sessions were 
led by teaching assistants (TAs). All TAs received the same training in leading these sessions at the beginning of the semester. During 
the semester, students worked in small groups of 2-3 during discussion and lab sessions. 

Our quasi-experiment took place in the lab session in week 4 of the semester. This lab session covered a topic related to molecular 
geometry: chemical isomers. Isomers are molecules made of the same atoms but differ in the spatial arrangement of their atoms. 
Differences in the atoms' spatial arrangements within molecules yield different chemical properties (e.g., melting point, optical 
activity). Instruction on isomers crucially relies on students' ability to translate between physical ball-and-stick models and wedge- 
dash structures (Fig. 1). 


4.2. Experimental design 


The chemistry course had 34 lab sections. Two experimental conditions received a framing prompt prior to drawing wedge-dash 
structures on the computer. Five lab sections (n = 94 students) were assigned to the model condition that received a model-focused 
framing prompt. Six lab sections (n = 125 students) were assigned to the draw condition that received a drawing-focused framing 
prompt. These sections were assigned to the experimental condition based on the availability of the chemistry building's computer 
labs. Further, we assigned sections to the model or draw conditions such that each TA was assigned to one of the experimental 
conditions. The remaining 23 sections were assigned to a business-as-usual control condition (n = 474 students) that did not receive a 
framing prompt or the technology-based collaboration script, but drew wedge-dash structures on paper, as is common practice in this 
course. Because students selected lab sections at the beginning of the semester to fit their schedule, we have no reason to believe that 
systematic differences exist between sections, except for the effect of the TA, which we accounted for in our analyses below. 

During the lab session, students in all conditions worked collaboratively in groups using the representations shown in Fig. 1 to 
solve a sequence of chemistry problems about isomers. Each isomer problem required students to construct a physical ball-and-stick 
model with their partner using a shared modeling kit and to translate the model into a wedge-dash structure that they drew in- 
dividually. Further, each problem contained conceptual questions that required students to use the representations to make sense of 
concepts related to chemical isomers. Below, we describe the differences between the two experimental conditions and the control 
condition. 


4.2.1. Experimental conditions: model and draw 

Students assigned to the draw and model conditions received an adaptive collaboration script that prompted them to discuss how to 
translate between physical ball-and-stick models and virtual wedge-dash drawings. As in our prior study (Rau et al., 2017), the script 
was implemented in an educational technology that contained a digital version of the isomer problems covered in the lab session: 
Chem Tutor. The Chem Tutor problems contained the same steps, same questions, and same molecules as the paper worksheet that 
was traditionally used in this lab session. 

As shown in Fig. 2, Chem Tutor instructed students to construct ball-and-stick models, draw wedge-dash structures within Chem 
Tutor, and answer conceptual questions using a drop-down menu. If students made an error, the adaptive collaboration script 
highlighted the part of the conceptual question or wedge-dash drawing that was incorrect and prompted students to discuss specific 
misconceptions that may have led to the error. The misconceptions were identified based on a cognitive model that underlies Chem 
Tutor. 
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Fig. 2. An example Chem Tutor activity. 


The difference between the model and the draw conditions regarded two framing prompts, shown in Table 1. Prompts were 
provided orally at the beginning of the lab session before they used Chem Tutor and as introductory text on the first page of Chem 
Tutor. Students in the model condition received prompts to “build and orient their physical ball-and-stick models” before constructing 
virtual wedge-dash structures in Chem Tutor. Students in the draw condition received prompts to “plan their wedge-dash structures on 
paper” before constructing them in Chem Tutor. 


4.2.2. Control condition: business-as-usual 

A “business-as-usual” control condition worked collaboratively without framing prompts or the script in Chem Tutor. Students 
also solved chemistry problems about isomers and used a shared modeling kit to construct the physical ball-and-stick models. 
However, they used a paper worksheet to draw wedge-dash structures and answer conceptual questions. The worksheet contained the 
same sequence of problem-solving steps, questions, and molecules. At the end of the 3-hr lab session, TAs collected the worksheets to 
provide written feedback on problem solutions and wedge-dash drawings in the following week's lab session. 


4.3. Assessments 


To assess students' learning of content knowledge, we used a pretest and posttest on isomers, evaluated in our prior study (Rau 
et al., 2017). A reproduction scale of the test assessed students’ ability to recall isomer concepts covered in the lab. A transfer scale 
assessed students’ ability to apply this knowledge to predict the stability of molecules. 

To assess students’ spatial skills, we used the Vandenberg & Kuse test for mental rotation ability (Peters et al., 1995). This test was 
evaluated in prior research (Peters et al., 1995) and has been used in prior research on the role of spatial skills for chemistry learning 
(e.g., Rau & Wu, in press; Stieff, 2007; Wu & Rau, 2018). 

To assess students' errors, we used log data from Chem Tutor to categorize errors related to either drawing or modeling. Draw 
errors involve inaccurate depictions of the molecule using wedge-dash conventions. Model errors involve inaccuracies relating to the 
orientation of the molecule. We computed the average number of draw and model errors for each student. 

To assess instructors' impressions, we conducted semi-structured interviews with four TAs: two who taught sections assigned to the 
draw condition (Daniel, Dylan) and two who taught sections assigned to the model condition (Michael, Macy). Macy also taught a 
section assigned to the control condition. All TA names are pseudonyms, selected to match TA's assigned experimental condition 
(draw, model). Each TA received $5 for participating in a 30-min interview. 


4.4. Procedure 


We conducted our study as part of an undergraduate chemistry course. Fig. 3 summarizes the procedure for each condition over 
time. A lecture in week 3 of the semester covered molecular geometry and chemical isomerism. Our experiment took place in the lab 
session in week 4, in accordance with students' typical course schedule. First, as the required pre-lab exercise, students completed the 
pretest and spatial skills test online before the lab session. 

Next, during their scheduled 3-hr lab session, students completed problems using the version of Chem Tutor or the paper 
worksheet that corresponded to their condition. In the experimental conditions, the first author provided the oral prompt that 
corresponded to the condition at the beginning of each session and then gave instructions on how to log in to Chem Tutor and use its 
interface. Throughout the session, the first author and research assistants were available for technical support, while the TA answered 
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Week 3 Week 4 Week 5 Week 6 


Lab: Draw on paper 
(business-as-usual) 


Activities O°" Lecture: Model Lab: Draw on Chem Tutor TA 

condition isomers prompt = Interview 

Draw Draw 
condition | =>) fa 
prompt | comesrsm ialarmintes 
Pretest; 
Assessments Spatial Posttest 
test 


Fig. 3. Timeline of the experimental procedure from Week 3 to Week 6 of the semester. 


conceptual questions. In the control condition, students received no specific prompts or introductions but were given a worksheet to 
complete the problems with lab partners, while the TA answered conceptual questions. 


Finally, students completed the posttest online as the required post-lab exercise at the end of week 4. We conducted interviews 
with instructors two weeks after the lab. 


5. Results 
5.1. Manipulations checks 


5.1.1. Excluded students 

Six students (n = 3 in draw condition and n = 3 in the model condition) were excluded because they failed to complete Chem 
Tutor problems. These students were absent or dropped the class during week 4 of the semester, as reported by their TAs. An 
additional 122 students were excluded because they failed to complete the pretest or posttest (n = 15 in draw condition, n = 16 in 
model condition, and n = 91 in the control condition). This yielded N = 565 students for our analyses (n = 107 in draw condition, 
n= 75 in model condition, and n = 383 in the control condition). 


5.1.2. Prior differences between conditions 

Because we used a quasi-experimental design in which students were not randomly assigned to conditions, we first checked for 
potential differences between conditions prior to the intervention. Table 2 shows the means and standard deviations of test scores by 
condition. 

A multivariate analysis of variance (MANOVA) with condition as the independent factor and test scores (reproduction pretest, 
transfer pretest, and spatial skills test) as dependent measures showed no significant differences between conditions on the re- 
production pretest (F < 1) or on the spatial skills test, F(1, 564) = 1.313, p = .270. However, there was a significant difference on 
the transfer pretest, F(1, 564) = 10.527, p < .001. Post-hoc comparisons showed that students in the draw condition had sig- 
nificantly higher scores than students in the control condition (p < .001). No other differences were significant at the pretest. To 
account for pretest differences, all following analyses used pretest scores as a covariate. 

Next, we checked for learning gains from pretest to posttest. A repeated measures ANOVA showed significant learning gains on 
the reproduction test, F(1, 564) = 150.923, p < .001, as well as on the transfer test, F(1, 564) = 78.235, p < .001. 


Table 2 


Means and standard deviation (in parentheses) of test scores by condition. Scores are calculated as a proportion of total 
possible correct answer, on a scale from 0 to 1. 


Condition 
Control Model Draw 
Spatial test -881 (.140) -875 (.132) .857 (.166) 
Reproduction scale 
Pretest -527 (.190) -515 (.170) -541 (.186) 
Posttest -651 (.195) -630 (.202) -628 (.210) 
Transfer scale 
Pretest -540 (.403) .654 (.397) .727 (.369) 
Posttest -745 (.356) -801 (.326) -764 (.375) 
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5.2. Differences in learning gains between conditions 


First, we investigated research question 1 (whether the model-focused and drawing-focused prompts are effective in enhancing 
content knowledge, compared to the control) and research question 1a (whether spatial skills moderate the effectiveness of the 
prompts). Because students were nested in lab sections, we used a hierarchical linear model (HLM) that can take into account nested 
sources of variance due to the fact that, for instance, students taught by the same TA may have more similar knowledge than students 
taught by different TAs. Specifically, we used an HLM that included a random intercept for TAs. In addition, the HLM included pretest 
scores as a covariate to control for pretest differences prior to the intervention, condition as the independent factor to test research 
question 1, and spatial skills and an interaction effect of condition with spatial skills to investigate research question 1a. 

On the reproduction posttest, there was no significant main effect of condition (F < 1) nor a significant interaction between 
condition and spatial skills (F < 1). On the transfer posttest, there was a significant main effect of condition on learning gains, F(2, 
547) = 5.445, p = .005, such that the model condition outperformed the control condition and the control condition outperformed 
the draw condition. This effect was qualified by a significant interaction between condition and spatial skills, F(2, 547) = 5.383, 
p = .005. To gain insights into the nature of this interaction effect, we split students into groups with low spatial skills (0-33 
percentile on the spatial skills test), medium spatial skills (34'"-66™ percentile), and high spatial skills (67'*-100" percentile). Post- 
hoc comparisons showed that the effect of condition was marginally significant only among students with low spatial skills, F(2, 
544) = 2.654, p = .071, but not among students with medium or high spatial skills, ps > .10. 


5.3. Differences in types of errors made between conditions 


Next, we investigated research question 2 (whether the model-focused and draw-focused prompts affect performance on the 
adaptive collaboration script) and research question 2a (whether spatial skills moderate performance). To this end, we compared 
the types of errors (model-errors, draw-errors) made by students in the model and draw conditions who used the adaptive colla- 
boration script (V = 180). Table 3 shows the means and standard deviations of the model-errors and draw-errors by condition and 
spatial skills. Table 3 also shows that, in total, students made more model-errors than draw-errors. 

Because only seven TAs taught sections that used Chem Tutor and conditions were assigned by TAs, we were unable to use the 
same HLM with a random intercept for TAs due to unacceptable bias in variance components (McNeish & Stapleton, 2016). 
Therefore, we used a MANCOVA with error types (model-error, draw-error) as the dependent measures, pretest scores as a covariate, 
condition as the independent factor (model-condition, draw-condition) to test research question 2, and spatial skills and an inter- 
action effect of condition with spatial skills to investigate research question 2a. 

There was a significant main effect of condition on error type, F(2, 176) = 4.969, p = .008, such that students in model condition 
made fewer model-errors and students in the draw condition made fewer draw-errors, F(2, 176) = 3.097, p = .048. This effect was 
qualified by a significant interaction between condition and spatial skills, F(2, 176) = 4.053, p = .019. As in the analysis above, we 
split students into groups with low, medium, or high spatial skills. Post-hoc comparisons showed that the main effect of condition was 
significant for student with low spatial skills, F(2, 176) = 3.538, p = .031, such that students with low spatial skills assigned to the 
model condition made fewer model-errors and those assigned to the draw condition made fewer draw-errors. There were no dif- 
ferences between conditions for students with medium or high spatial skills (ps > .05). 


5.4. Exploration of TA impressions 


To explore research question 3 (instructor impressions of the framing prompts), we analyzed TA interviews in a qualitative 
manner. Specifically, we summarized TA responses with respect to students' interaction with models and drawings during the lab 
session and how Chem Tutor helped or hindered interactions with models and drawings. Below, we describe TA responses on the use 
of models and the use of drawings using direct quotes from TA interviews. 


5.4.1. TA impressions on interactions with models 

Regarding how students interacted with ball-and-stick models, some TAs noted that the students often stopped building models. 
Particularly, the two TAs who taught sections assigned to the draw condition, Daniel and Dylan, respectively stated that many 
students “stopped building models after the heptane isomer” halfway through the lab, and that “not every group, I think, without 


Table 3 
Means and standard deviations (in parentheses) of model and draw errors by condition and students with low (0-33rd percentile on the spatial skills 
test), medium spatial skills (34th-66th percentile), and high spatial skills (67th-100th percentile). 


Model Errors Draw Errors 

Model Condition Draw Condition Total Model Condition Draw Condition Total 
All students 23.8 (18.9) 28.9 (23.4) 26.8 (21.8) 13.4 (8.6) 12.7 (9.3) 12.9 (9.0) 
Low spatial skills 27.7 (20.5) 30.4 (26.6) 29.4 (24.3) 15.8 (8.2) 11.0 (7.0) 12.8 (7.8) 
Medium spatial skills 28.1 (22.8) 34.0 (23.9) 31.6 (23.4) 15.4 (9.4) 16.4 (12.8) 16.0 (11.5) 
High spatial skills 16.8 (10.8) 22.8 (18.4) 20.2 (15.9) 9.7 (7.2) 10.9 (6.1) 10.4 (6.5) 
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prodding would've um been building everything ... I don't think they believe it's worth their effort.” Dylan believed that models 
helped students understand the concepts: “students had to work with the models physically as well to really get it, I think just doing it 
on the screen, they were struggling until they really sat down with the models and did it that way.” The TAs assigned to the draw 
condition found that students were often not engaging with the physical models, even though the models helped students learn the 
concepts. 

The TAs indicated that Chem Tutor helped engage students with the models, particularly when compared to the typical lab session 
that only used a paper worksheet. Michael, who taught sections assigned to the model condition, stated that students “can learn more 
deeply when they try to convert the modeling kits [...] to answer the questions on the computers. But I mean if you just need to write 
down the structure on the, on the paper, some—I will say, some students have no idea what the stuff means.” Michael believed that 
students may not learn as much from only using paper (i.e., the control condition) compared to using the technology to engage with 
models (i.e., model condition). Macy, who taught a section assigned to the control condition and a section assigned to the model 
condition, felt that the feedback from the Chem Tutor script engaged students with the models, in comparison to the paper worksheet: 
“T did like the immediate feedback that [students] would get so like, ‘This is the wrong structure. How do you fix it?’ I liked that 
‘cause like, I can't really do that when they're doing the when they're um, writing on a sheet.” All TAs found that the feedback from 
the script “filter[ed] out, like, the really low-level questions” (Dylan), which allowed TAs to “address questions more easily and 
rapidly” (Macy). The technology also allowed TAs to “build the model with [students] and spend some more time with them. Let the 
other people who are not having as much trouble do their thing and help them a little bit as needed” (Dylan). The TAs reported that 
feedback from Chem Tutor seemed to have helped engage students with the models and reduced the number of questions that the TAs 
typically answer, which allowed TAs to engage their students with models. 

In sum, TAs articulated that students, particularly those in the draw condition, may not have sufficiently engaged with models. 
Chem Tutor seemed to have provided some instructional support via immediate feedback that helped students address questions with 
models. This freed time for TAs to focus students on models and build them together. 


5.4.2. TA impressions on interactions with drawings 

With regard to how Chem Tutor affected learning with wedge-dash drawings, all four TAs expressed the positive sentiment that 
Chem Tutor “helped students learn the wedge-dash notation” and “in general, I think it's good for students” (Michael). They ex- 
plained that the technology helped students engage deeply with the content because “it was cool. It was a good way to, like, better 
visualize the molecules” (Macy) and “it kind of inspires them to try a little harder, I think, to learn how to draw these structures” 
(Dylan). TAs noted that, as a result, students asked more conceptual questions, for instance, about “why one drawing worked and one 
didn't” (Daniel). Overall, TAs viewed the technology as a good way to help students engage with wedge-dash drawings. 

In addition to drawing in Chem Tutor, all TAs recommended that students draw on paper, either as intermediate or final drawings 
on paper worksheets or in students! lab notebooks. For example, Macy proposed that students “write [molecules] down [on paper] 
before or after they do the computer.” When comparing her students assigned to the control condition vs. the model condition, Macy 
stated that “the ones that had to draw it [on paper], I think, had a little bit better understanding than the ones who did it on the 
computer, um although they both like, were able to make the molecules” She believed that “drawing it out by hand [...] is a bit more 
helpful,” but also “[t]hat just could be [her] bias ‘cause [she] like[s] writing things out.” All TAs expressed a similar sentiment: 
“conceptually, if I want to understand something, I usually write it out [on paper]” (Macy). Dylan additionally explained that 
drawing helps prepare students for the exam: “I'm thinking that eventually [students] still have to each draw it on the test, even just 
to get that—just getting that muscle memory, I think they should draw all the molecules.” Hence, all TAs suggested that students 
draw out certain molecules on paper and show them to TAs, even if some parts of the lab activity involved drawing on the computer, 
as Michael suggests: “I will still prefer the student answer the questions on the computer but maybe ... for example, when you 
assemble like a very huge molecule [...] a student can easily draw this molecule on the lab notebook.” Only Macy, upon reflection 
near the end of her interview, realized that “all the isomer questions were more easily understood by the people who did the 
computer-based section, thinking back to the worksheet that [she] graded.” Until she recalled the evidence from her grading, she 
believed the opposite: drawing on paper is more effective. Thus, she initially recommended that students draw on paper, as suggested 
by the other TAs. 

In sum, the TAs articulated that Chem Tutor helped students engage with drawings. Further, they viewed drawing as an important 
learning and thinking strategy for them, and by extension, for their students. Thus, they recommended prompting students to 
generate more intermediate or final drawings on paper, even if students also drew within the technology. 


5.5. Post-hoc exploration of students' collaborative interactions 


Our findings suggest that framing prompts changed the nature of students' interactions with the representations and their peers. 
However, our quasi-experiment did not directly measure the effects of the adaptive collaboration scripts on collaboration quality 
because our main goal was to investigate the effects of framing practices on students' learning of content knowledge. To explore such 
possible effects, we conducted a post-hoc analysis of collaboration rate, as measured by the proportion of problems that students 
completed within five minutes of their partners. We found a marginal effect of condition on collaboration rate, t(177) = 1.758, 
p = .081, such that partners in the model condition collaborated at a higher rate (M = 79.7%, SD = 28.0%) than partners in the draw 
condition (M = 71.6%, SD = 31.3%). All TAs reported that even when “students sometimes were on different problems, [they] still 
helped each other” (Daniel). 
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6. Discussion 


This quasi-experiment investigated the effects of framing students' interactions with an educational technology that helps them 
translate among multiple visual representations. Specifically, we tested whether prompting students to focus on physical models or on 
generating intermediary drawings was more effective (research question 1), and whether spatial skills moderated the effect of 
framing prompts (research question 1a). We found that model-focused framing yielded higher learning gains on a test of chemistry 
knowledge transfer, compared to non-scripted collaborative translation in a control condition. By contrast, drawing-focused framing 
yielded lower learning gains on the transfer test than the control condition. These effects were marginally more pronounced for 
students with low spatial skills. However, we found no effects on reproduction of chemistry knowledge. 

To investigate possible mechanisms underlying these effects, we compared rates of model-errors and draw-errors between the 
model and draw conditions (research question 2), and tested whether spatial skills moderated these effects (research question 2a). We 
found that the model-focused prompt reduced students' errors related to modeling, and the drawing-focused prompt reduced errors 
related to drawing. These effects were more pronounced for students with low spatial skills. Moreover, we found that students overall 
made more model errors than draw errors. 

Then, we explored instructors' impressions of the model-focused and drawing-focused prompts (research question 3). In contrast 
to our results, instructors seemed to favor the practice of generating intermediate drawings on paper: the framing practice that was 
less effective than the control condition. 

Lastly, we explored rates of students' collaboration on problems as a potential mechanism underlying the effects of the experi- 
mental conditions. We found that students in the model condition collaborated at a marginally higher rate than students in the draw 
condition. TAs also suggest that students often helped their partners, even if they were working on different problems. 

In sum, our results suggest that model-focused framing was more effective than drawing-focused framing in terms of enhancing 
knowledge transfer. We propose two possible reasons why this framing was effective, especially for students with low spatial skills. 
First, model-focused framing may direct students’ attention to the more difficult aspect of the translation task. In both experimental 
conditions, students made more model errors overall, suggesting that they struggle more with models than with drawings. Recall that 
model-focused framing prompted students to focus on constructing and orienting their models before drawing within the technology. 
This may have helped students orient their models to address the more difficult aspects of the translation activity: spatially aligning 
and mapping visual features between representations. Further, focusing on rotating the physical model before drawing may be 
particularly beneficial for students with low spatial skills who struggle with mental rotation because they can focus on spatially 
orienting the model to make it easier to map it into the 2D plane when generating the wedge-dash drawing. 

Second, focusing collaborative interactions on physical models may increase collaborative activity on shared representations. 
While the drawings in the technology were generated separately, the ball-and-stick models were constructed using a shared modeling 
kit. Focusing the interactions on a shared object may have encouraged students to collaboratively construct the model and discuss 
relevant concepts. This aligns with our exploratory finding of collaboration rate that students in the model condition tended to work 
together to complete problems, more so than students in the draw condition. Students in the model condition likely worked together 
to construct and orient the model, which allowed students to discuss concepts during the translation task. In particular, students with 
low spatial skills may have benefited from discussing how to orient the model with their partners, which then facilitates its trans- 
lation to a drawing. 

On the flipside, our results also suggest that drawing-focused framing was less effective than unscripted collaboration, especially 
for students with low spatial skills. We propose three reasons why this framing was ineffective. First, recall that drawing-focused 
framing prompted students to generate an intermediary drawing on paper before drawing within the Chem Tutor technology, where 
they received feedback on their drawing. Students may not yet have the skills to draw wedge-dash structures on paper without 
feedback. Given that this is difficult for students, asking them to generate additional drawings may increase cognitive load. Because 
the translation task already imposes higher cognitive load for students with low spatial skills, drawing-focused framing is more likely 
to result in cognitive overload for these students, thereby impeding their learning. 

Second, drawing-focused framing may reduce students’ reflection on translations among the representations when the colla- 
boration script prompted them to do so. Because students had already generated a drawing on paper when they translated it into the 
drawing on Chem Tutor, they may have focused more on copying from the paper to the technology, instead of reflecting on how to 
translate 3D physical models into 2D drawings. 

Third, drawing-focused framing may decrease collaboration. Observations showed that students tended to generate drawings 
individually on paper. This may have reduced discussion between partners to help each other understand how to translate between 
representations and possibly resulted in the marginally lower rate of collaboration among partners, compared to the students in the 
model condition. Further, because drawings were not shared but “belonged” to individual students, students may be less likely to 
make changes to their partner's drawings. This may have particularly disadvantaged students with low spatial skills who would have 
benefited from help in aligning physical models with paper-based drawings. 

The most surprising result was that the TAs recommended drawing-focused practices, in contrast to our findings that this framing 
practice was least effective. It is likely that instructors have an expert blind spot (Nathan & Petrosino, 2003); they view drawing on 
paper as useful because they already have the skills to draw, but forget that students do not. In our study, the TAs expected that “a 
student can easily draw the molecules.” Only one TA realized that drawing on paper may not be as effective as providing Chem Tutor 
with the model-focused prompt—and she only realized this at the end of her interview. Thus, she and the other TAs recommended 
that students generate more drawing on paper to learn content knowledge. Until they reflect on students' difficulties, TAs may not 
realize their blind spot and thus may choose to frame students’ interactions on generating drawings, which increases difficulties for 
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students. 

Finally, our results showed no differences between conditions on the reproduction test. This finding aligns with our prior study 
and other work, which shows that collaborative activities are less effective for simple concepts, than for complex concepts (Rau et al., 
2017; Kirschner et al., 2010). Complex concepts may be well-suited for collaborative efforts because students can help each other 
explain concepts and resolve divergent views (Mullins, Rummel, & Spada, 2011). However, for simple concepts, collaboration in- 
creases task difficulty by requiring students to coordinate information and discuss errors that they can individually address. Hence, 
our results suggest that adaptive collaboration scripts may be particularly beneficial for enhancing learning of complex concepts, not 
simple concepts. 

In sum, our study suggests that relatively slight differences in how instructors frame students' interactions with an educational 
technology can lead to drastically different results for its effectiveness, to the extent that one framing practice (i.e., the draw con- 
dition) resulted in the collaboration script being less effective than a non-scripted control condition, whereas another framing practice 
(i.e., the model condition) resulted in the script being more effective than the control in terms of enhancing transfer of content 
knowledge. Notably, both experimental conditions used the same educational technology; the difference between the conditions lied 
solely in the way the technology was introduced to students by providing framing prompts. Surprisingly, we found that instructors 
may not choose the most effective framing to help their students learn with educational technologies. 


6.1. Theoretical and practical implications 


Our results extend theory on students’ learning with multiple visual representations by identifying two mechanisms that may 
account for the effectiveness of collaborative activities with multiple representations: focusing students' interactions on physical 
models or on generating intermediary drawings. The findings from our study suggest that, especially for students with low spatial 
skills, focusing shared attention on physical models is a mechanism that may enhance students' learning. By contrast, focusing on 
intermediary drawings is a mechanism that might hinder students' learning. 

These theoretical contributions yield practical recommendations for the design of technologies with collaboration scripts for 
STEM instruction. We recommend that support for collaboration scripts focus students’ interactions on shared physical models rather 
than on individually owned intermediary drawings. Further, we caution instructors against encouraging students to generate in- 
termediary drawings when translating among representations, especially if they have low spatial skills. 

Additionally, our results offer new explanations for prior research on collaborative learning that have yielded conflicting results 
on the effectiveness of collaboration scripts (Magnisalis et al., 2011; Vogel et al., 2016; Walker et al., 2009). The framing used in prior 
studies with collaboration scripts may have reduced the effectiveness of collaborative activities. Our study shows how framing 
practices, even when implemented as simple prompts provided at the beginning of a class period, can significantly impact the 
effectiveness of technology-based collaboration scripts. 

This finding has practical implications for instructors' framing of students' interactions with educational technologies in the 
classroom. Instructors may fail to recognize which aspects of a task may be most difficult for students and where collaborative 
interactions could most benefit them. By focusing students’ collaborative interactions on the difficult aspects of the task, they may 
enhance their learning. Conversely, by focusing students on other aspects of the task, they may inadvertently increase cognitive load 
and deter collaboration, which may reduce learning, especially for students with low spatial skills. 

Finally, our results have practical implications for increasing equity in STEM classrooms. We found that students with low spatial 
skills were particularly sensitive to framing practices and that framing prompts focused on generating intermediate drawings before 
drawing in an educational technology may increase disparity in the STEM classroom. Instructors value the practice of generating 
drawings because it helps them learn, but such instructional support entrenches traditional practices that may continue to exacerbate 
inequality in STEM. Focusing on traditional practices may not provide the support needed by students with low spatial skills, who are 
less likely to enter STEM careers without support to build their spatial skills (Uttal et al., 2013; Wai, Lubinski, & Benbow, 2009). Our 
work contributes to our understanding of how instruction with technologies can help students overcome difficulties with multiple 
representations in STEM and how best to design technologies with effective, equitable instructional practices. 


6.2. Limitations and future directions 


Our findings should be interpreted in light of the following limitations. First, causal inferences from quasi-experiments are limited 
because non-random differences between conditions and unmeasured differences may have affected the results. Hence, an experi- 
ment with random assignment of individuals to conditions should replicate our results. 

Second, while our results suggest that increased collaboration rate might account at least in part for our findings, we did not 
directly measure collaboration, and indeed TA interviews indicate that the log data approximation may not capture when and how 
students collaborated. Therefore, our discussion on collaboration is tentative without additional data on the level and quality of 
students' interactions. Future work should directly assess collaboration by examining how and when students help their partners with 
the translation tasks. Particularly, future work should compare the effects of partners with similar spatial skills vs. different spatial 
skills, to understand the effects of collaboration for students with low spatial skills. Such research may yield recommendations for 
how instructors should create student groups and encourage students' collaborative interactions with educational technologies. 

Third, our experiment did not measure students' cognitive load as they interacted with the technology. Because drawing has been 
shown to increase cognitive load (Schwamborn et al., 2011), differences in cognitive load may explain why the adaptive colla- 
boration script was effective when framed by the model-focused prompt and not effective when framed by the drawing-focused 
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prompt. To maintain equivalence between conditions, we did not include cognitive load measures. However, future studies should 
include measures of cognitive load and test whether differences in cognitive load mediate the effects of framing practices on learning 
outcomes. 

Finally, our quasi-experiment investigated whether framing practices, as implemented by prompts provided to introduce an 
educational technology that contains an adaptive collaboration script, were more effective than a control condition without prompts 
or the technology. Our quasi-experiment did not include a control condition that used the same framing practices without an edu- 
cational technology or an educational technology without framing prompts. Therefore, we cannot make inferences about the ef- 
fectiveness of the framing practices independent of an educational technology that supports collaborative learning. Future studies 
should address this limitation by isolating the effects of framing practices and educational technologies. 


7. Conclusion 


Our quasi-experiment with undergraduate chemistry students investigated which instructional framing practice enhances stu- 
dents' learning from an educational technology that helped them collaboratively translate among multiple visual representations. 
Results showed that framing students' interactions by prompting them to focus on physical models enhanced learning gains on a 
transfer test, whereas framing interactions with a prompt that focused on generating intermediate drawings on paper reduced learning 
gains, compared to a business-as-usual control condition that received no technology-based support. Our results suggest that model- 
focused framing helped students utilize shared resources and collaborate on the difficult aspect of spatially aligning 3D models with 
2D drawings to translate between the representations, especially for students with low spatial skills. Further, these results contradict 
the impressions of instructors, who play a crucial role in framing students’ interactions with technologies. In sum, this study suggests 
that the way instructors frame students’ interactions with educational technologies can significantly affect their learning with 
multiple visual representations. 
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